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ABSTRACT 

Under Subtitle B Part I, Serve-America, of the 
National and Community Service Act of 1990, $1.1 million was awarded 
to fund 44 school-based service-learning programs in Pennsylvania. A 
study collected data on program implementation and impact on 
students, Serve-America coordinators, and supervisors during visits 
to 43 sites and at meetings with site coordinators. Disseminator 
(schools and community-based organizations) and minigrant 
(service-learning) programs were funded. Five variables were 
evaluated as indicators of implementation: organizational commitment, 
integration, parental support, advisory councils, and materials. Ten 
percent of all programs benefited from all five variables being 
strong and reinforcing. More than 50 percent were implemented as 
planned. Generally, students expressed satisfaction, but their 
dissatisfactions included lack of support from school staff and other 
students, not enough time at sites, and insufficient clarity 
concerning the program and their roles. The program increased 
coordinators' personal satisfaction and excitement, but added stress 
and work. Service-learning had both a positive and negative impact on 
supervisors. Stressors were longevity and institutionalization of the 
programs, loss of funding, and involvement of all students. 
Challenges were identified, and recommendations were made to develop 
a vision statement and a work plan. Three elements of an evaluation 
system were already in place: empowered evaluation, tracking and 
monitoring, and mentoring/consultant infrastructure. (YLB) 



ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft 

* Reproductions supplied by EDRS are the best that can be made * 
Vc from the original document. ,f 

ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft 



§ The Pennsylvania Serve-America Grant: 



£ Implementation and Impact 

S Year 1 



Prepared for PennSERVE: 
The Governor's Office of Citizen Service 
1304 Labor and Industry Building 
Hanisburg, Pennsylvania 17120 



U » OCFAirTMENT Of EDOCATMDN 

O^iCe cx €<3uc*liOoit R«M«rch aryJ JmofOv«n»#flt 

EDUCATIONAL RESOURCES INFORMATION 
f CENTER (ERIC) 

-ectfiveo t'om cn« o#rion Of organization 
or<Qirvaling a 
2 Mine cnanges nave Dean made 10 improve 

■"DfOOuCltOn Quality 

• Po«nt»o> view Of Op<niOn» Hated mt hit dOCU- 
-n«ni oo not necessarily <apre*ent official 

"ER" DOSitiGn Of ooncy 



July, 1993 



PERMISSION TO REPRODUCE THIS 
MATERIAL ^AS BEEN GRANTED BY 



C 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER «ERlC) ' 



Carl I. Fertman, Ph.D. 
Irving Buchen, Ph.D. 
Joanne Long, MPH 



The Pennsylvania Service-Learning 
Resource and Evaluation Network 
School of Education 
University of Pittsburgh 
5D21 Forbes Quadrangle 
Pittsburgh, PA 15260 
(4I2>648-7196, Fax (412) 648-7191 



Heric 



Pennsylvania Service-Learning 
Resource and Evaluation Network 



The Pennsylvania Service-Learning Resource and Evaluation 
Network at the University of Pittsburgh is working with all the 
Pero SERVE Serve-America programs. The network is a source of 
communication, support, resources, and guidance. Its philosophy is 
to work with people by providing systematic data of a wide variety 
of types that help inform them about things they care about. The 
primary goal of the network is to help each PennSERVE Serve- 
America site evaluate its service learning program. 

For more information please contact Carl I. Fertman, Ph.D., 
Director, The Pennsylvania Service-Learning Resource and 
Evaluation Network, School of Education, University of Pittsburgh, 
5D21 Forbes Quadrangle, Pittsburgh, PA 15260, (412) 648-7196, Fax 
(412) 648-5911 



The Pennsylvania Service-Learning Resource and Evaluation Network is 
supported by a grant from PennSERVE: The Governor's Office of Citizen 
Service, 1304 Labor and Industry Building, Harrisburg, Pennsylvania 17120, 
Telephone 717-787-1971, Fax 717-787-9458 



Executive Summary 



"The Pennsylvania Serve- America Grant: Implementation and 
Impact" is a report of the PennSERVE: The Governor's Office of 
Citizen Service, first year activity under the Commission on 
National and Community Service, Public Law 101-610, Subtitle B 
Part I Serve- America award, to involve school-aged youth in service 
to the community through school and community based 
organizations. 

The focus of the report is the implementation and impact of service- 
learning programs in Pennsylvania. All data for the current report 
were collected during visits to 43 PennSERVE Serve- America sites in 
the months of April, May, and June 1993 and at meetings on June 2 
and 8, 1993 with all site coordinators. A separate report of 
participant hours and service activities is in preparation consistent 
with the national evaluation information system guidelines. 

The report provides answers to two questions: 

** Did PennSERVE accomplish its goal of implementing service- 
learning programs in schools and community based organizations? 

** What are the impacts of the programs on students., program 
coordinators, and supervisiors at the end of year one? 

The report concludes with a summary of programmatic issues and 
concerns the schools and community based organizations will face 
during the next 12 months and the identification of a number of 
critical issues and opportunities PennSERVE will encounter in the 
year ahead. 

The overall conclusions of the report: Pennsylvania has successfully 
implemented its Serve- America grant program; 90% of its programs 
were implemented as planned; the programs have had an often 
dramatic but always positive impact on students; an intervention has 
occurred in the routine of the classroom, in the predictable operation 
of the school environment and in the day to day workings of the 
community where the full effect is yet to be fully felt and evaluated. 
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Section 1 Background 



The National and Community Service Act of 1990 (Public Law 101-610, as 
amended) created the Commission on National and Community Service to 
provide program funds, training, and technical assistance to states and 
communities to develop and expand opportunities for national and 
community service. Under Subtitle B Part I Serve-America, the Commission 
awarded $16.3 million to support new or ongoing efforts to involve school- 
aged youth in service to the community through school and community 
based organizations. The primary focus of Serve-America (representing at 
least 60% of its funding) is the development of school-based service-learning 
strategies that integrate service into the academic and extra-curricular life of 
school for grades K-12. In Pennsylvania $1.1 million was awarded to fund 44 
programs statewide and to develop a service-learning support and evaluation 
network to support these programs. 

The current report focuses on the implementation of the 44 programs and 
their impact on students, Serve-America program coordinators, and 
supervisors. All data for the current report were collected during visits to 43 
PennSERVE Serve-America sites in the months of April, May, and June 1993 
and at meetings on June 2 and 8, 1993 with all site coordinators. A separate 
report of participant hours and service activities is in preparation consistent 
with the national evaluation information system guidelines. 
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Section 2 Implementation 



"When people find out how good these kids are and what they can 
do, they're goin want to know how they can have more of them do 
more." (Community member - Elizabeth Forward) 

"Service, what service, this is a science class" ( 9th grade girl - 
Moshannon Valley School District) 

"I asked to talk to six students involved with service-learning. I was 
clear, I thought, about the number I asked to speak with. I promised 
to be brief: maybe ten minutes, at the most fifteen minutes. In walked 
24 students. We talked the entire period" (Evaluator field notes - 
Derry Area School District) 

Did PennSERVE accomplish its goal of implementing service-learning 
programs in schools and community based organizations? The answer is yes. 

Two types of programs were funded: disseminators and mini-grants. 
Disseminator grants went to schools and community based organizations 
(CBO's) with demonstrated experience, organizational commitment, and 
community support. Twelve disseminator grants were funded. Mini-grants 
were to initiate service-learning programs. Prior experience was not a 
prerequisite for funding. Interest and organizational commitment were. A 
total of 34 mini-grants were funded. The breakdown of the programs by 
setting, location, and focus are displayed in figures 1 and 2. 





Disseminator grants 


Mini-grants 


Total Grants 


12 


32 




School 


10 


27 


Community Based 
Organization 


2 


5 




Rural 


1 


16 


Suburban 


4 


5 


Urban 


7 


11 



Figure 1 Grants by setting and location 
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Disseminator grants 


Mini -grants 


Elementary School 


1 


3 


Middle School 


1 


6 


High School 


10 


23 



Figure 2 Grants by school focus 



Five variables were identified for examination and evaluation: 
organizational commitment, integration, parental support, advisory councils, 
and materials. 

Organizational commitment is the thrust of a program. It is the tangible and 
spirited guidance, reinforcement, assistance and support provided to the 
service-learning program by an organization. It signals the priority within 
the organization; and frequency, in the culture of the school, that service- 
learning is perceived as a strategy for change. 

Integration is penetration. It is concerned with the degree to which service- 
learning is infused into learning through the curriculum, staff preparation 
and support, and academic credit. It also defines the degree to which it 
becomes part of the organizational mission. 

Advisory councils are forums for advocacy. Required wisely as part of the 
grant, their effectiveness is measured by the number of its members, number 
of meetings, and member's organizational diversity. 

Parents are the learning partners of service-learning. They play many roles. 
They develop youth; they support the development of service opportunities. 
They are role models and beneficiaries of service. Parent support is assessed 
through parent awareness, involvement, and communications. 

Materials represent the goods and services produced by service-learning. The 
quantity and quality of program products is staggering. Moreover, programs 
also access materials and other resources from various support systems and 
thus multiply the results. Networking draws together the National Center for 
Service-Learning and School Change, Pennsylvania Institute for 
Environmental and Community Service-Learning, Generations Together, 
Center for Intergenerational Learning and VYTAL Program. 

System wide variables, such as, the lateness of funding, slow contracting 
process, poor PennSERVE communications and service-learning's high 
political profile, surprisingly, did not differentially affect programs. The 
distribution of frustrations and struggles is universal and always associated 
with new initiatives and creative work. 
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Differences did exist between programs in terms of implementation. Program 
implementation falls into one of four categories: Yes Plus, Yes, Yes Minus, 
No But. Although the categories are subjective they serve as a vehicle to 
help understand the implementation process. 

Yes Plus implementations represent ten percent of both disseminators and 
mini-grant programs. All benefit from all five variables being strong and 
reinforcing. All are differentiated by very strong organizational commitment. 
In these programs, service-learning was integrated as a teaching strategy; 
have large, diverse advisory councils; parental involvement is extensive; and 
numerous materials are created and acquired. 

Such programs capture the spirit of the school. An example is Keystone 
Elementary Program which built the school around service. The Elizabeth 
Forward High is permeated by service-learning. These programs have gone 
beyond their stated goals. 

More than 50% of the programs are categorized as Yes and were 
implemented as planned. They enjoy strong organizational commitment. 
They are strong in the areas of integration and accessing and creating 
materials. The advisory councils of the programs have diverse membership. 
The councils are active. But the parent component of these programs is weak. 
Parents are interested, but parent awareness, involvement, and 
communication tire scattered or occasional. 

Yes Minus programs as a group experienced an organizational struggle 
which partially interfered with their implementation. They account for 30 % 
of the programs. An example is a change in the coordinator. A person who 
had no previous service-learning experience suddenly found himself or 
herself in charge of the program as an additional responsibility. As a 
consequence, the programs' integration and advisory councils suffered, as 
also did parent awareness, involvement, and communication. Nevertheless 
these programs typically have maintained links to materials and support, and 
have produced materials. 

The final programs are No But, They comprise 10% of the programs. These 
programs are uniformly weak or very weak across all five variables. The 
organizational commitment initially present at the writing of the grant was 
not sustained with the receipt of the grant. Without that central 
commitment, the programs' integration, advisory council formation, 
parental involvement, and materials development suffer. When No But 
programs succeed, they tend to be programs built on the strength and skills of 
one person. Without the organizational commitment and advisory council 
support to confront all the bureaucratic issues, these programs are always 
fragile. They are handicapped before they start. They struggle. 
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Figure 3 Categories of implementation by five variables. Rating scale is VS = 
Very Strong, S = Strong, W = Weak and VW = Very Weak 



The five variables are indicators of implementation. Figure 3 shows the 
rating for each variable by implementation category. Of the five, obtaining 
and maintaining organizational commitment was a priority for each 
program. Creating and accessing materials was the second priority. 
Integration, advisory councils, and parents are the next areas of focus within 
the five variables. 

Program implementation is a process carried out over time. The effect on the 
process of any one variable's strength or amount or the combination of 
variables is at this point unknown. Future reports v/ill address this issue. 
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Section 3 Impact 



"I didn't know math was so much fun" (Student, Kennedy Crossan 
Elementary School) 

"I learned I can lead" (Student, Kensingston High School) 

"The students take pride in their projects and do not vandalize or 
destroy them " (Teacher, East Lycoming School District) 

"We are modeling future schools" (Assistant Superintendent, 
Scranton School District) 

"There is great potential for school change and to change the 
'culture of learning' (Coordinator, Furness High School) 

"Older citizens often, for the first time, appreciate youth" (Advisory 
Council Member, Derry Area School District) 

The three groups most involved in and most impacted by service-learning 
are students, program coordinators, and supervisors of Serve-America 
program coordinators. These three groups have the most contact with the 
programs. Service-learning, good or bad, has touched their lives. Students are 
the life blood of service-learning. They are the reason it exists. Coordinators 
are the front-line of service-learning. Their job is to make it happen. They 
wrote the grants, traveled to conferences, made the connections, and continue 
to be the leaders. Supervisors provide the support and link the program to 
the bigger organization. 

The current report focuses on the impact service-learning has on students, 
coordinators, and supervisors Future reports will focus on the impact of 
service-learning on advisory council members, teachers, community based 
organization staff, and community members. 



Service-learning exists for students. They are its life blood. Students involved in 
service-learning span elementary school, middle school and high school levels. 



Academic learning and personal development: these represent the core of the 
programs 1 impacts on students. Transformative experiences have served to empower 
students, to bring life back in their school days and to recreate a desire to learn anc 1 
succeed. Students have fun. 

It is an incredibly frustrating task to attempt report the positive experiences noted by 
students within the scope of this document. Comments of students themselves most 
accurately represent this phenomenon and give powerful insight into the workings of 
service-learning. 

One student's comment captures the twofold benefit of the program by describing the 
paradoxical nature of "giving" as benefiting both the recipient and volunteer. These 
students have learned that they can have a voice in the community, can have an 
intergenerational effect upon how youth is perceived, can attain peer status in an 
adult world of work environment, and can become functioning and valued members 
of their communities. Students expressed wonderment at the appreciation and 
respect shown them at their sites. More concrete benefits reported by students were 
learning their academic subjects in a new and challenging way, affirmation of 
practical skills, exposure to the real world of work and opportunities for career 
exploration and career testing, as well as material to present on college and 
employment applications. Students learned to function in teams and developed 
leadership characteristics. 

Although expressions of satisfaction were many, students dissatisfactions occasionally 
surfaced. These expressions included lack of support from school staff and other 
students, not having enough time at the sites, and insufficient clarity of program and 
their roles. Revealingly, during student evaluations at one site, students expressed 
frustrations with the lack of respect from adults and staff, with a stereotype of not 
being able to do the work, with not being recognized for their service and similar 
issues. When asked if these dissatisfactions were enough to have them think about 
quitting, the response was encouraging and revealing as service-learning appeal: 
"NO, you have to deal with them and can't quit!" In summary, the overwhelmingly 
positive experiences of participation far outweighed any problems students faced. The 
opportunity to translate themselves and to test themselves in an adult real world 
situation offered students an opportunity unique to their experience. 

In the upcoming stages of service-learning, students would like to see the program 
expanded to include more students, more classes, more sites, extended student roles, 
and to begin earlier in the year. The issue of voluntary versus required participation 
was also reported as needing examination. Many students reported lack of teacher 
support for their participation; efforts to orient teachers and to better connect service- 
learning with the curriculum might minimize this aspect. To meet these requests, 
greater preparation and time are needed. 
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Coordinators are the front-line of service-learning. Their job is to make it 
happen. The program impacted on their lives by increasing personal 
satisfaction and excitement, but also adding stress, and work. 

For 25% of the coordinators, the program impact was overwhelmingly 
positive. They participated in the grant writing and design and had 
experience with grant management and implementation. They had 
anticipated many of the initial details. They were already setup and operating. 
They were able to harness the energy from the initial excitement of learning 
and used the grant to propel themselve 3 beyond their anticipated goals. They 
were empowered leaders. 

The impact of the additional stress and work for half of the coordinators was 
considerable. Their lives were more complicated and demanding. They 
reported many personal satisfactions and rewards. However, at times, these 
were overshadowed by the paperwork, bureaucratic red tape, and feeling 
overwhelmed by the grant. They got worn down by the work. For the most 
part, they did not anticipate the amount and pace of the work the grant 
required. 

A final 25% of the coordinators were impacted by events beyond their control 
and which tended to draw them away from the grant activities and rewards. 
For example, the lawsuits in one district kept the coordinator focused on legal 
issues, which were interesting, but interfered with her duties. Some were 
waiting for job reclassifications. In these cases, they had anticipated the job of 
coordinator as part of their responsibilities in a new position. However, they 
found themselves doing their old jobs plus being the grant coordinator. 
Likewise, a few coordinators were assigned to the job late in the school year, 
with no prior knowledge of service-learning. In most cases, their initial 
expectations were unrealistic. Some coordinators were poorly received by the 
systems they were trying to impact. This left them frustrated. 

At the end of year one, the anticipated turnover rate of coordinators is 20%. 
Review of the first year data revealed that the coordinator position in many 
schools and community organization is evolving. It is not seen as being the 
responsibility of a particular group or department. It is a responsibility that 
can be reassigned. Likewise, it can be anticipated that a number of 
coordinators would change due to teacher leaves, sabbaticals, illness and new 
job assignments. For example, one coordinator is now a long term substitute 
teacher awaiting a permanent assignment. Anticipated budget reductions will 
also impact some positions. 

Coordinator turnover reinforces the fact that service-learning is a "new 
program. 1 * Staff movement is common in all new human services and 
education programs. More established programs experience less personnel 
movement. 
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Supervisors are pivotal. They support the link to the bigger organization. 

Supervisors typically are principals, superintendents, executive directors, or 
board presidents. Although generally they are positive about service- 
learning, in most cases their knowledge of service-learning is limited. 
Competition for their time and energy is keen. The red tape and bureaucracy 
associated with any grant means more work for them. 

Service-learning has had both a positive and negative impact on supervisors. 
For example, supervisors from both disseminator and mini grant sites 
reported mixed progress, but with more positive progress overall. Funding, 
bureaucracy, and resistance from some students, faculty, and parents were 
noted as frustrating factors. On the other hand, all of the supervisors 
expressed satisfaction with the warm community response toward the 
programs. Enhanced school image, positive feedback, and high levels of 
enthusiasm were all noted as initial responses to the programs. Although 
many supervisors reported that they were not surprised ^y anything that 
transpired, others were pleasantly surprised by the high levels of community 
support and student involvement. 

The negative impact of service-learning on the supervisors is demonstrated 
through their concerns and anxieties about the longevity and 
institutionalization of the programs. Many of the supervisors expressed 
extreme reliance upon the coordinator for the ongoing success and 
functioning of the program. Many expressed complete trust in their 
coordinators when asked about program monitoring. For example, some 
stated that they would let the coordinator do his job and asked to be informed 
only about statistics, accounting, and major problems. 

The second highest stressor was the loss of funding for the program. The 
supervisors stated that the service-learning programs could not exist without 
it. The supervisors had reported spending a considerable amount of time 
doing project budgeting and accounting. 

The third highest stressor was involving all students in service-learning, 
particularly those educationally/ academically at-risk. Negative student 
behavior and the potential for a tarnished school image were identified as 
possible reasons for program failure and negative community relations. 
Overall, the supervisors were concerned that the teachers, parents, students 
and the community might lose interest in service-learning in the future. 
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Section 4 Frogrammatic concerns and issues 



Based on the analysis to date, it is predicted that programs will face a series of 
challenges during the upcoming twelve months. 

The good news is that the programs are already working to address the 
challenges. The extent of their plans and preparations is largely a function of 
their level of development. 

* Service-learning is young. Some programs trace their roots back to the 1988; 
others to 1992. CBO's and schools are still trying to figure out what it is. At 
the same time, they are attempting to differentiate it and determine its 
relation to concepts such as outcome based education, community education, 
apprenticeships, school reform, vocational education, field trips, and class 
projects. The concept of service-learning as a teaching strategy that every 
teacher might use is evolving. Frequently it is focused on at-risk students and 
drug prevention, which is too narrow a focus and minimizes the full 
potential of service-learning as a teaching method. 

* Eight key ingredients of service-learning programs Evaluation tells us that 
commitment and interest are not enough. Eight key ingredients are 
necessary to make a service-learning cake. They are students, adults, advisory 
council, service activity, linkages, community support, parents, and 
communication. Realistically, programs need to choose one or two 
ingredients and try to work on them over time. Additionally each ingredient 
has several elements. For example, the students' elements are preparation, 
reflection, monitoring and celebration. 

* Student success measures: - The ultimate determination of whether service- 
learning succeeds as school reform is the degree to which it can be shown that 
service -learning impacts student success measures. Each program needs to 
identify a concrete way to measure the effects of service-learning. 

* Coordinator's roles - Coordinators' roles vary. Depending on the urogram, 
the coordinator's role could be that of a service club advisor, classroom 
teacher using a new teaching strategy called service-learning, grant 
administrator, lead teacher supporting colleagues interest in service-learning, 
in-service instructor, or director of a school volunteer action center. 
Coordinators need help in figuring out their roles. 

* Problems: As programs develop problems, will develop. It is natural and 
expected. At the June 2 and 8 meetings for example, time was spent discussing 
problems the coordinators had encountered. Furthermore, time was 
dedicated to brainstorming alternatives and solutions. Problems encountered 
implementing the programs included teacher support, service-learning 
advocacy and marketing, fiscal issues, time constraints, and communications. 
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Section 5 PennSERVE Implications 



Given all this information, what can PennSERVE do? Several actions by 
PennSERVE are recommended to help progress be achieved. To a large degree 
the actions are natural consequences of overall system development. 

1. * Vision statement: PennSERVE has to anticipate the needs of the programs 
it funds and evaluates. It needs to develop a vision statement, a five year 
plan of growth and development to identify the critical issues and areas of 
future leadership and support. 

2. * Work Plan: The work plan will evolve as an outgrowth of the vision 
statement. The work plan includes all of PennSERVE's activities as well as 
specific goals and expected outcomes for each title activity (Serve Corp, Serve- 
America). As part of the work plan, strategies to address critical areas would 
be developed and shared, for example, linking the grantees within title 
activity and across title activity or providing support and technical assistance. 

Clearly the Serve-America program is successful. The competition for 
resources and technical assistance will intensify. Expectations for the program 
are already increasing. PennSERVE's provision of communication, guidance, 
resources, advocacy, and coordination can provide the underpinning to 
continue the growth begun by the positive impetus of initial 
implementation. 
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Section 6 Conclusions - Multiple Measures — Multiple Impacts 



Although this report will be supplemented by additional analysis and 
evaluation, and although this is only the first year of operation of the grants, 
certain conclusions seem inescapable. 

A. Timing 

If service-learning is an idea whose time has come, it is also a time whose 
idea has come. Service-learning has emerged at a time when there is a 
historical need for educational change, Moreover, service-learning tends to 
be reinforcing and linking in its relationships, principally in its ability to 
reconnect the school to the community and the community to the school. 
Above all, service-learning seeks to structure a partnership between self 
esteem and intellectual development that is at the heart of lifelong learning. 

To be sure, panaceas for K-12 have come and gone; fads have surfaced and 
disappeared within one year. Is service-learning any different? Hopefully, 
yes, because it is broad-based, not singular, in focus and because it comes at a 
time when only broad-based constituencies can restore confidence in public 
education. 

B. Not Just Change - But Transformation 

If there is a danger of excess advocacy for . orvice-learning, it is that of 
overselling — of appearing to sound like a revivalist preacher talking about 
being born-again. And yet again and again that is what evaluation uncovers 
and discovers. Classroom and curricula seem to be reinvested with reality; 
teachers find special satisfaction in managing, not dominating, the learning 
process; administrators for the first time in many years strut proudly before 
public groups, school boards, senior citizens, even city councils because now 
schools do not just take but give back to the community; citizens previously 
down on teenagers are often now their biggest boosters; service clubs 
dominated primarily by the retired and elderly now sense a new lease on the 
future working with young students; community agencies severely 
underfunded suddenly find a new transfusion of volunteers to help staff 
their understaffed programs. And last, but certainly not least, the students 
themselves often experience an emotional and academic growth spurt in one 
year (often after many years of slumber) defying all predictioi The recurrent 
phrase heard again and again from teachers, parents, administrators, even 
friends is, " I can't believe it is the same kid." 
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C The Challenge of Evaluation 



Service-learning does so much, involves so many, is simultaneously so 
immediate, yet long term, that it poses a formidable task for evaluators. No 
single measure will suffice; even multiplying the number of measures begs 
the question, for still they may miss all the interaction between the parts. In 
addition, because most community evaluators know little about schools and 
most school evaluators know little about communities a common interface is 
lacking. At PennSERVE, we are persuaded that service-learning is breaking 
new ground and requires a new evaluation system and approach. The 
components of the new evaluation system are emerging, and although what 
that finally may be is not yet fully in hand, three key elements are clear and in 
place. 

1. Empowered Evaluation 

The evaluation process must be administered and delivered closer to the 
operational level. Moreover, teachers and students need to be not just the 
objects of evaluation but the managers of it. But for empowered evaluation 
to take hold, the evaluation process must be, on the one hand demystified 
and, on the other, user friendly. That has been one of the principal goals of 
The Pennsylvania Service-Learning F ^source and Evaluation Network. 

2. Tracking and Monitoring 

The dilemma is one of richness. The programs themselves are complex. 
They involve numerous projects, hundreds of students, thousands of hours, 
different community and agency players, etc. In addition service-learning 
itself is essentially developmental and distributes its growth in so many 
different directions, often simultaneously, that it resembles the reverberating 
effect of the stone striking the v ater and sending out wider and wider circles. 
The problem then is a mix of the quantitative and qualitative. Detailed 
monitoring is necessary juj>t to keep track operationally of all that is going on. 
At the same time, monitoring needs to be done to identify the branch points 
of change, intervention and transformation. Moreover, since many students 
increasingly perceive service-learni ig as a way of testing future careers and as 
a leg up for competing for scholarships and admissions to prestigious schools, 
jervice-learning has also to find ways of documenting and validating its own 
activities — developing in effect a service-learning transcript. PennSERVE 
and The Pennsylvania Service-Learning Resource and Evaluation Network 
have encouraged the use of electronic software to monitor and track the 
activities of ihe program u/ng students to provide input into the system 
(they want to run it anyhow). The goal is to generate a disc for each student 
that records all the service-learning activities. Essentially, the student can 
send a copy of that disc with his application to college. 



3. Mentoring / Consultant Infrastructure 

Because of : :he complex multiplicity of service-learning projects (it is very 
much like a school within a school) and further because it turns the 
classroom around to face out wardly to the larger community, the service- 
learning coordinator (who basically is a classroom teacher transformed 
overnight into an administrator), often is overwhelmed. That coordinator 
needs massive support, especially during the first year. The Pennsylvania 
Service-Learning Resource and Evaluation Network provides that support. 
Materials, meetings, phone conversations and all site visits provide each 
coordinator with his or her own consultant/broker and mentor. Every 
conceivable problem is discussed: the budget, problems with the principal, the 
roster, the schedule, logistics, difficulties with other teachers, classroom 
materials, operational issues, etc. In addition, the coordinator is made aware 
of other consultative individuals and organizations available to make his or 
her life more interesting and manageable. The net result is that coordinators 
are surrounded by a support system that will net let them fall off the edge. At 
the same time, they are connected to a network that enriches and extends 
their project. 

Above all, the The Pennsylvania Service-Learning Resource and Evaluation 
Network consultants in their site visits experience and evaluate the program 
in miniature. Typically, the evaluator meets with administrators, parents, 
members of the advisory council, students, community agencies, sometimes 
even the Mayor and the City Council. As a result, the consultant replicates the 
multiple constituencies of the coordinator and thus is able to minister 
comprehensively to the coordinator's reality. The foreground is thus 
matched by the background. Behind every grant stands a strong commitment 
of expertise, savvy and leverage committed to the success of the coordinator 
and the program. Given the range and complexity of service-learning, that 
kind of support is minimally a norm. Perhaps, it should be a norm for all 
teachers. 



"14 *>\ 



Appendix A Evaluation site visit planning letter to coordinators 

Robert Howard, Service-Learning Coordinator 

Service-Learning 

Service, PA 12345 

Dear Robert: 

As promised, here is the next step in the PennSERVE evaluation: member 
profile and site visit information. 

I ask that you complete the member profile. Please answer all the profile 
questions and collect the requested materials (Don't worry, there are no right 
or wrong answers.) Our goal is to get a picture of what is happening in each 
program and to help you think about how you want to evaluate your 
program. Please make a copy of the profile for your records. 

The site visits are planned for April, May, and June. The major focus of the 
visit is to review the completed member profile. A typical visit is 3 to 4 
hours. Reviewing the profile takes about two hours. We also are required to 
meet with the advisory committee (45 minutes), your supervisor (30 
minutes) and 8 to 10 students in a small group for 30 minutes. During each 
meeting individuals will complete a survey. 

From your supervisor, we are interested in gauging organizational support 
and interest. With the advisory committee, we will explore developmental 
issues, service activities, collaboration, and linkages. The time with students 
will focus on the value of service. 

You will be telephoned by either Irv Buchen, Joanne Long, or myself, Carl 
Fertman, during the next few weeks to set up a day to visit. 

We will also have an evaluation meeting at the PennSERVE Conference on 
April 19. Please reserve the date. The mini-grants and disseminator grants 
will have separate meeting times. It will be a chance for us to talk and answer 
questions. Likewise, we will meet in Philadelphia and Pittsburgh in early 
June. Call if you have questions. Thanks for your help and cooperation. We 
look forward to visiting you. 

Best wishes, 

Carl I. Fertman, Ph.D. 

cc: Irv Buchen(215) 635-8933 
Joanne Long (412) 648-7188 

enclosures: site profile 
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